Configure an Export Definition - File Transfer

The use of a Secure FTP server, an Amazon S3 Bucket, or a Google Cloud Storage (GCS) Bucket is a common method of extracting data from the Loyalty platform, and transmitting it to an external site where it can be picked up for use in another platform or system.

File transfer types of Export Definitions can be used for extracting Members, Offers, Offer Responses, Campaigns, or Places.

This article assumes that you have already created the Export Definition, either by creating it from scratch, or by copying an existing Export Definition.

To configure an Export Definition:

1. Select Admin from the top navigation bar, then select Integrations > Exports from the side navigation menu. The Exports screen is displayed.

2. On the Exports screen, search for the desired Export Definition (see Search for an Export Definition for more information).

3. Click the name of the desired Export Definition. The Export Definition Details screen is displayed.

The Export Definition Details screen is split into multiple steps, identified by a series of numbered items across the top of the screen. You can complete these steps (described below) in any order you like. The Next button at the bottom of the screen navigates to the next step in the sequence, and the Back button navigates to the previous step.

Step 1: Filter

The Filter step allows you to select the records included in the export file. Depending on which Resource you selected for this Export Definition, the platform provides several different methods for selecting these records: using a simple filter, using a Segment (Members only), or using an analytics Query.

Select Records with a Simple Filter

If you selected Person, Offer Response, Campaign, Offer, or Place as the Resource for this Export Definition, the platform lets you define a simple selection filter comprising one or more logical Rules. The use of a filter is optional; if you don't define a filter, the platform will export all records of the selected Resource type.

1. At the top of the Export Definition Details screen, click Filter.

2. To create a new Rule, click the Add icon.

3. From the first drop-down menu, select an Attribute.

4. From the second drop-down menu, select an operator.

Note: The LIKE operator allows you to search for part of a string using "%" as a wild card character. For example, if you wanted to export all Member records that have "gmail.com" as their email address domain, you would select LIKE as the operator, then enter the value of "%@gmail.com."

5. In the text field, enter a value.

6. Repeat the above steps as needed to define additional Rules.

7. If your filter has multiple Rules, use the toggle buttons to define the logical operator used to join the rules: AND or OR.

8. Optionally, to remove a Rule, click the Remove icon next to it.

9. Optionally, you can also organize Rules into Rule Groups. A Rule Group allows you to build more complex selection criteria. To create a Rule Group, click Group. The platform creates a Group, which is represented visually as a nested box with its own operator toggle and Add icon. Within this Group box, click the Add icon to define the Rule (or Rules) using the same process as described above. Use the toggle within the Group box to define the logical operator for all Rules within that Group. By clicking the Group button inside the Group box, you can create "nested" Groups.

Note: The selected operator within a Group applies to all Rules in that Group. If you need the relationship between some Rules to be AND, and others to be OR, you would need to use multiple Groups, each with their own separate operator.

10. Click Save. Or click Next to move to the next step in the Export Definition screen.

Select Members with a Segment

If you selected Person Segment as the Resource, the platform lets you choose a Segment. All the Members selected by this Segment will be included in the export. By using a Segment, you can define more complex selection criteria than the simple filter described above.

1. At the top of the Export Definition Details screen, click Filter.

2. From the Segment drop-down menu, select the desired Segment.

3. Optionally, to refresh the Segment before exporting, toggle Refresh Segment to On.

4. Click Save. Or click Next to move to the next step in the Export Definition screen.

Select Records with a Query

If you selected Query as the Resource, the platform lets you choose an analytic Query that defines what records to export.

1. At the top of the Export Definition Details screen, click Filter.

2. From the Bigdata Query drop-down menu, select the desired query.

3. Click Save. Or click Next to move to the next step in the Export Definition screen.

Step 2: Map & Transform

The Map & Transform step is used to define the data map for the export file. The data map allows you to identify which field in the source table should be written to which column in the export file. You can either define the data map manually, or upload a file of column header names to define the data map.

Note: This step is not displayed if you selected Query as the Resource, as the Query itself will define the columns to be exported.

Manually Define a Data Map

To manually define the data map:

1. At the top of the Export Definition Details screen, click Map & Transform.

2. Click Add Row (or select Add Row from the Add drop-down menu).

3. From the Field Name drop-down menu, search for and select the desired Attribute.

4. Check Required if the Attribute must be populated to be included in the export (records with NULL values will be skipped). Otherwise, leave Required unchecked if having a NULL value in this field is acceptable (rather than skipping the record).

5. Optionally, to perform a data transformation on an Attribute, click Add. From the pop-up menu of available options, click the desired Data Transformation. Repeat this step as needed to add additional Data Transformations.

6. In the Destination field, enter the column header name as it will appear in the export file.

7. Repeat the above steps to define additional Attributes to export. Optionally, to remove an Attribute, click the Delete icon next to it.

8. Click Save.

9. Click Next to move to the next step in the Export Definition screen.

Upload Headers

Instead of manually defining the data map as described above, you can upload a file of column header names to use as the data map. This file must be in .csv or .txt format. The platform will attempt to automatically match each header name in the uploaded file to an Attribute.

1. At the top of the Export Definition Details screen, click Map & Transform.

2. Click Upload Headers (or select Upload Headers from the Add drop-down menu). The Upload Headers pop-up window is displayed.

3. Click Choose File. Browse to and select the desired file, then click Open.

4. From the Delimiter drop-down menu, select a delimiter character (Comma or Tab), or select Custom and enter the custom character.

5. Click Upload. The Map & Transform screen is populated with the column header names from the uploaded file. The column header names are displayed beneath Destination, in the same sequence as they appear in the uploaded file. If the platform was able to match a header name with an Attribute, the Field Name drop-down menu is populated with that Attribute name.

6. For the columns where the platform found a match, verify that the match is correct. If the match is incorrect, or if the platform couldn't match to an Attribute, select the desired Attribute from the Field Name drop-down menu.

7. Optionally, to perform a data transformation on an Attribute, click Add. From the pop-up menu of available options, click the desired Data Transformation. Repeat this step as needed to add additional Data Transformations.

8. Optionally, beneath Destination, edit the column header names. These values represent the column names as they will appear in the export file.

9. Check Required if the Attribute is required.

10. Optionally, if you need to add an Attribute beyond what was included in the file of column header names, select Add Row from the Add drop-down menu. The platform adds a new blank row to bottom of the screen, where you can select the Attribute and enter the name of the column. Optionally, to remove an Attribute, click the Delete icon next to it.

11. Click Save.

12. Click Next to move to the next step in the Export Definition screen.

Step 3: Destination

The Destination step is used to define the destination of the export file. This step requires that you have a Connection Profile defined. For more information on creating a Connection Profile, please see Connectors.

1. At the top of the Export Definition Details screen, click Destination.

2. From the Connection Profile drop-down menu, select a Connection Profile.

3. In the File Path field, enter the path to the folder on the FTP server, Amazon S3 Bucket, or Google Cloud Storage Bucket where you want to save the export file.

4. In the Filename field, enter the name of the export file.

Note: The file should be named so that it is unique each time it exports, so a new file doesn't overwrite a previous file. The best practice recommendation is to add a timestamp to the filename, such as {(yyyyMMdd)}.

5. From the Delimiter drop-down menu, select a delimiter character (Comma or Tab), or select Custom and enter the custom character.

6. Optionally, from the Text Qualifier drop-down menu, select a text qualifier character (double quotes, single quotes, or pipe), or select Custom and enter the custom character.

7. From the Line End drop-down menu, select the method used to mark a line break in the export file: LF (Line Feed) or CRLF (Carriage Return and Line Feed).

Note: The Item field has been deprecated and should be left blank.

8. If you want the export file to include a column header row, select Yes from the Include Headers drop-down menu.

9. From the File Encoding drop-down menu, select an encoding method (the default method is UTF-8).

10. Optionally, to export a Control File along with the export file, enter the desired Control File name in the Control File Name field. The Control File contains the expected record count of the export file. The filename must have an extension of either.control or .ctrl .

11. Optionally, to PGP-encrypt the export file, select the desired Encryption Key from the Encrypt With drop-down menu.

12. If you want to allow the platform to export empty files, select Yes from the Allow Empty Files drop-down menu.

13. Click Save.

14. Click Next to move to the next step in the Export Definition screen.

Step 4: Schedule

Export Definitions can be executed on-demand (see Execute an Export Definition for more details on that process).

In addition, you can create a schedule that executes the export once at a specified date / time, or on a recurring basis. These schedule options are described below.

Execute Only Once

To schedule an Export Definition to execute once at a specified date / time:

1. At the top of the Export Definition Details screen, click Schedule.

2. Toggle Enabled to Yes.

3. From the Recurrence drop-down menu, select Once.

4. Click into the Execute Once On field. A calendar pop-up window is displayed. Select the desired date when you want the export to execute.

5. Within the calendar pop-up window, click the clock icon. Define the time when you want the export to execute.

6. From the Time Zone menu, select the time zone to use when determining the time to execute.

7. Click Save.

Executing on a Recurring Schedule

To schedule an Export Definition to execute on a recurring frequency:

1. At the top of the Export Definition Details screen, click Schedule.

2. Toggle Enabled to Yes.

3. From the Recurrence drop-down menu, select Repeat.

4. Next to Schedule Composer, select an interval:

   • 15 Minutes

   • 30 Minutes

   • Hour: Select the minute past the hour.

   • Day: Select the time of day.

   • Week: Select the day of the week, and the time of day.

   • Month: Select the day of the month, and the time of day.

   • Year: Select the day, the month, and the time of day.

5. As you configure the interval above, the CRON Expression field is automatically populated with the associated CRON expression. Optionally, to manually edit this expression, check Advanced Mode.

6. Optionally, using the From and To fields, define the export schedule begin and end date. If you leave these date fields blank, the schedule will run indefinitely.

7. From the Time Zone menu, select the time zone to use when determining the time to execute.

8. Click Save.

Email Notifications

You can configure the Export Definition to trigger success and / or failure notifications to one or more email recipients.

1. Optionally, if you want to send error notifications, toggle Error to Yes, then enter the email addresses for the notification recipients.

2. Optionally, if you want to send success notifications, toggle Success to Yes, then enter the email addresses for the notification recipients.

3. Click Save.